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college preparatory curriculum with a distinct career theme; and partnerships 
with employers, communities, and higher education. Findings indicated academy 
curricula did not hinder students from completing courses required for 
graduation; any effects of test taking on academy programs (or vice versa) 
were overshadowed by the generally negative opinion and by the disruptions to 
school operations those testing requirements produced; and teachers and 
students found these tests burdensome, students sometimes resisted taking 
them, and skepticism about their value and their validity was broadly 
expressed. Steps suggested for academy programs were to give increased 
attention to building and maintaining a coherent and stable educational 
experience for students in each academy; to explore more ways to demonstrate 
their students' unique attainments; and to find ways to ensure their students 
succeed in the standardized testing environment. (YLB) 
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I. 



Introduction 



Over the past two decades, the Career Academy model has spread widely across the 
landscape of secondary education in the United States. Launched initially as a modest program 
variation within high schools in Philadelphia, there were an estimated 3,000 Academies 
established in districts throughout the country by 2000. 

Support networks have grown up around this expanding population of programs. The 
Career Academies Support Network, at the University of California, Berkeley, and the National 
Academies Foundation in New York City work on a national level. Local organizations, such as 
the Philadelphia Academies, Inc., assist programs in individual communities. A national 
evaluation project, sponsored by the U.S. Departments of Education and Labor, and a group of 
foundations, is underway to provide long-term evidence of the effectiveness of nine Career 
Academies. 

The popularity of Career Academies reflects several trends. The most important, perhaps, 
is the widespread interest in raising the quality and achievement of public education, which arose 
in the early 1980s with publication of A Nation at Risk. The quality of public education has been 
a prominent national topic, and strategies that show potential for its improvement — especially 
in the context of urban education — have drawn attention. 

Career Academies appeared to show promise, and they possessed other intriguing 
qualities as well. The best of them featured active partnerships with the private sector, and thus 
resonated with the burgeoning policy interest in school-to-career strategies. Though they did not 
embrace traditional vocational learning strategies, they nevertheless emphasized the real 
connection between learning and earning. Academies also appealed to a range of student ability 
levels, and stressed not immediate employment but academic achievement and post-secondary 
education. In addition. Academies — which, by design, are small in scale — fit naturally with 
the movement of the past two decades to “deconstrucf’ large and impersonal high school 
settings. The creation of individual “houses” and small learning communities to personalize the 
school experience dovetailed well with the Academy approach. 

Academy programs, therefore, had appeal both on their own merits, and also in the larger 
context of school reform. School districts or individual principals interested in adding an 
innovative program within a stable, established setting could do so. Likewise, those seeking 
broader strategic reform could use the Academy model as a tool for bringing about change 
within the individual school or, over time, within the larger school district. 

The widespread adoption of Academy programs has naturally generated attention and 
concerns among educators and policymakers. With the current stress in U.S. education on 
curricular rigor and accountability. Career Academies have raised a new set of questions in the 
minds of educators and policymakers. Two in particular are the subject of this paper: 

1 . Do Academy programs help or hinder students to complete courses that are required 
for graduation from high school or entry into college? 
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2. Do Academies enhance student readiness, preparation, or performance on the high- 
stakes tests they increasingly are required to take? 

These questions emerge in current discussions of public education for several reasons, 
among which the theme of accountability is paramount. Current debate about education is 
frequently couched in terms of performance measurement, and the resulting calls for national 
standards and testing have found receptive audiences in states and local communities. States 
have been particularly energetic in implementing new achievement testing programs and putting 
greater emphasis on programs already in place. 

Added to this is the growing importance of post-secondary education of all kinds. While 
the traditional emphasis on four-year college attendance remains strong, there is a heightened 
interest in seeing more (if not all) youngsters continue schooling for career-related reasons. An 
ever-increasing demand for a skilled and educated workforce has reinforced the message 
students are receiving that a high-school diploma no longer guarantees long-term success in the 
labor market. Participation in community college, professional institutes, and other educational 
tracks are viewed as all-but-essential complements to work after high school. 

The focus on standards in combination with the need to encourage higher levels of 
student educational attainment has brought renewed attention to the high school curriculum. 
Many argue that it is no longer sufficient that schools focus solely on core skills, but instead 
should help students build a foundation for their education after graduation. This new emphasis 
manifests itself, for example, in the incorporation of prerequisite courses for admission to state 
universities into secondary curricula and a mounting pressure for students to complete an even 
fuller roster of required courses. 

At the same time, schools are called upon to demonstrate that what they teach actually 
“works” as measured increasingly by student performance in standardized tests. Whether test 
scores are ultimately found to be an adequate — or accurate — gauge of school performance, 
there can be no question that, in today’s environment, they are taken seriously and that they 
command a substantial commitment of resources. 

How these broader education issues play out in the context of Academy programs has 
emerged as an important concern, and one that this paper addresses specifically by asking: Do 
Academy programs, with their tailored curricula, dilute the content of the required curricula, and 
are Academy students, who follow this alternative course of study, less prepared to score well in 
standardized tests? 

A. Plan of this Study 

This study represents an initial attempt to answer qualitatively the two issues posed 
above. Based on an examination of current practices in Academy programs in six school 
districts, it is part of a project jointly administered by Manpower Demonstration Research 
Corporation (MDRC) and the Career Academy Support Network (CASN), administered by the 
University of California, Berkeley. A related purpose of the project is to help inform the work of 
CASN, which provides technical assistance to Academies across the country. 
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Though more modest than a quantitative approach based on a statistical examination of 
actual student course completion, grades and test scores, this qualitative study gathered evidence 
through site visits, interviews, and review of available reports and documents to address these 
questions: 

□ How are Academy students informed and counseled about course requirements? 

□ How is student completion of courses — especially district-wide required courses — 
monitored? 

a Do conflicts arise between Academy curriculum and district curriculum, and if so, 
how are they resolved? 

u What preparation or assistance is available to Academy students for taking high- 
stakes tests, and how does it differ from that available to the general student 
population?' 

The descriptions and discussion that follow are derived from observations and interviews 
at the schools themselves. Interviews were conducted with the following groups: 

□ Principals 

a School counselors 

□ Academy coordinators 

a Teachers 

□ Students 

Interview protocols were prepared for each of these groups, with the aim of eliciting their 
distinct perspectives on the questions noted above, and a “building” perspective was adopted in 
this study to develop information within a given high school in a school district. The information 
reported here reflects actual practice (and attitudes) in individual buildings, even though what 
goes on within the school might vary somewhat from the stated policies of the school districts of 
which they are a part. Thus, for example, under Academy program policy students are “cohort- 
rostered” to take courses with their Academy peers to the extent possible. 

In individual schools, however, teacher availability or individual student course 
preferences (such as a desire to take a particular elective course, or an Advanced Placement 
course) may limit the extent to which cohort rostering is actually practiced. 

Indeed, a broad finding of this study is that building practices, especially practices 
adopted by individual high schools to accommodate the conflicting demands of individual 
programs (including Academies), testing requirements, graduation requirements and individual 

'Tables A and B, included in the Appendix, provide a more formal statement of the investigative framework 
employed. 
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student needs have as much, if not more, effect on the operation of Academy programs as the 
programs do on course-taking and test preparation. 

B. The Schools and the Programs 

For the study, a total of seven schools in six school districts were visited.^ Table 1 
provides summary detail about each school and its history and experience with Academy 
programs. A mix of schools was selected, with both long-established and newly formed 
Academy programs. One purpose in doing this was to develop some comparative insight on how 
new programs met district-wide curriculum and testing requirements while seeing to their own 
development and operations. (Though the schools are identified by name in tables, they remain 
anonymous elsewhere in the report in order to preserve confidentiality and to ensure that 
interviewees would feel free to offer candid observations without risking potential 
embarrassment to the institutions.) 

Course Taking, Test Preparation, 
and Career Academy Programs 

Table 1 

Schools Participating in the Study 



Academy 

Experience Academy 

(Years) Programs 



City 


High School 


<3 


3-8 


>8 


Examined 


All-Academy?* 


Miami 


Miami Beach 






X 


1 of 4 


Fall 2001 


Oakland 


Fremont 




X 




2 of 6 


No* 




Castlemont 




X 




3 of 5 


No* 


Philadelphia 


Lincoln 






X 


2 of 4 


Fall 2001 


St. Louis 


Roosevelt 


X 






2 of 4 


No* 


San Jose 


Oak Grove 


X 






2 of 9 


Yes 


Seattle 


Ballard 


X 






1 of2 


No 



*A1I-Academy schools have elected to enroll each student in an Academy program. 
’School plans to adopt the all-Academy model, but timetable not set. 



Two of the seven schools — Lincoln High School, in Philadelphia, and MBHS, in Miami 
— were selected specifically because they had stable, longstanding histories of operating 
Academy programs, and both have elected to become “all-Academy” schools in the fall of 2001. 
This means that all students attending those schools will participate in an Academy program. The 
decision by a school to go “wall-to-wall,” as will be seen, has consequences for (and is affected 
by) the issues of curriculum and testing that are the main focus of this study. 

Of the remaining five schools, two were in the early stages of Academy development. 
Ballard High School, in Seattle, had just three programs, one of those in existence less than a 
year. In San Jose, Oak Grove High School had adopted a school-wide “pathway” strategy, with 
each of the career-related pathways having some of the characteristics of an Academy, but only 



^An additional school was also visited to guide development of research questions and protocols. 



having two fully developed Academy programs. The remaining three had somewhat longer 
experience with Academy programs and were moving toward “all- Academy” status.^ 

All of the schools are located in urban areas and have economically and racially 
heterogeneous student populations. All were also in some state of organizational transition, 
resulting largely from the introduction of new Academies or from the impending decisions to 
move to an “all-Academy” set-up."* 

The similarities and differences among the schools notwithstanding, there are three key 
characteristics define the Career Academy model: 

1 . A “small learning community" established within a high school consists of students 
attending most classes together and teachers working as a team to provide an 
integrated and coherent educational experience. The objective is to have students 
identify closely with their curriculum, remain in contact with teachers over a longer 
period of time, and, as a consequence, increase the likelihood that their education will 
be personalized in ways that large high schools often do not permit. 

2. A college preparatory curriculum with a distinct career theme mixes both career- 
related courses and academic courses that meet graduation requirements. The 
curriculum’s career theme is not intended to be a traditional vocational program but is 
meant, instead, to provide a tangible context for learning, and a framework for 
integrating course material from such apparently uncormected topics as math and 
English. High quality standards are maintained to assure that Academy students who 
complete the program are prepared to enter college or other post-secondary settings. 

3. Partnerships with employers, communities and higher education help bring an 
Academy’s locally selected theme to life for students through active contacts with 
partners in industry and other relevant institutions. Academies establish relationships 
with employers relevant to their career theme. Relationships an Academy cultivates 
with employers relevant to its career theme may lead to industry instructors, 
internships, informational visits to area firms, and sometimes to part-time 
employment (paid and unpaid) for students. At the same time, students are made 
familiar with educational requirements for success in that career area, and encouraged 
to continue learning beyond high school. 

With their blend of one-on-one attention, stimulating curriculum, tangible employer (and 
community) cormections, and high standards. Academies are expected to boost the motivation of 
students who participate. In turn, the students would be expected to attend school more regularly, 
work harder, achieve better grades, graduate, and be more likely to go on to college. 



^This last group of schools was operating in near-crisis conditions, which themselves affected the viability of 
existing Academies and raised questions about their likely success in becoming “wall-to-wall” Academy schools 

‘'in two of the study schools, a new approach to ninth grade also was being adopted, a “Success Academy” 
based on a model developed by the Center for Research on Education of Students Placed at Risk (CRESPAR). This 
change entails segregating ninth-graders from the larger student body and immersing them in small learning 
communities. This change, too, added to the organizational flux the schools were experiencing. 
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Though open to all high school students, Academies have been viewed as an especially 
useful option for students who are academically at risk. By providing a more coherent and 
personalized instructional setting oriented around careers and work, but nonetheless holding to 
rigorous academic standards. Academies can appeal to young people disengaged from traditional 
education. They can also provide a viable pathway through secondary school and increase access 
to post-secondary education with career-related employment. 

In undertaking this study, it should be stressed, the integrity of individual Academy 
programs and their success in embodying these three characteristics were not directly examined. 
Nonetheless, as will be made clear, some findings regarding course selection and rostering do 
have a bearing on how robustly the Academy model was implemented in the study schools. In 
particular, several points of concern around the issues of cohort rostering and modifications to 
the content of required graduation courses will be discussed later. 

II. Curriculum and Course Taking 

Are Academy students, involved in a program with its own curriculum, might be at 
greater risk of failing to complete required course requirements for high school graduation or 
college entrance? 

This risk could manifest itself in any of several ways. The curriculum developed in an 
Academy could simply fail to include some required courses. Academy programs are built 
around courses connected to their career theme, and those career-focused Academy courses 
would simply displace courses required for graduation. A second cause of concern is that 
Academy courses might not be countable toward meeting college entry requirements. A third 
reason might be that completing the Academy curriculum could leave too little time for required 
coursework to be completed or present rostering conflicts that would prevent students from 
completing curriculum requirements. 

Site visits at Academy schools lead to these initial observations. First, while it is useful to 
think of Academy programs as distinct and in some ways separate from the rest of a school’s 
teaching and other activities, in practice Academy programs and courses are regarded 
administratively in much the same way as other courses. In all of the schools visited, student 
progress toward completion of graduation-required courses was a basic and pervasive concern. 
In all cases, the curriculum was consciously designed to span both the thematic aims of the 
Academy and the school district requirements for graduation. 

Those requirements, with some variation, are similar in all the study schools and in the 
school districts, in general. Table 2 displays the current standards for key graduation subjects.^ 
These were starting-points for Academy curriculum in all the study schools, with courses in 
English, math, social and natural science constituting the core graduation requirements for all 
students. 



^Other requirements, not shown in the table, included courses in physical education, fine arts, and elective 
courses. 
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Course Taking, Test Preparation, 
and Career Academy Programs 

Table 2 



Credits Required for Graduation from High Schools, by Study Site 





English 


Mathematics 


Social 

Science 


Science 


Foreign 

Language 


Total Credits 


Miami 


4 


3 


3 


3 


0 


. 24 


Oakland 


4 


3 


3 


3 


1 


23 


Philadelphia 


4 


3 


3 


3 


0 


21.5 


St. Louis 


4 


4 


3 


4 


0 


22 


San Jose 


4 


2 


3 


2 


1 


21 


Seattle 


4 


2 


3 


2 


0 


20 



In practice, the Academy curriculum is comprised of a combination of core courses and 
specially designed electives. The electives provide the Academy program’s thematic element. 
Drawn from curricula in Miami and Philadelphia, Table 3 illustrates what an Academy sequence 
might look like in a given year. 

Course Taking, Test Preparation, 
and Career Academy Programs 



Table 3 

Examples of Academy Course Rosters from Two Programs 



Miami Beach High School 
Academy of Travel and Tourism 
Sample 1 Grade Roster 


Lincoln High School (Philadelphia) 
Horticulture Academy 
Sample 1 P*’ Grade Roster 


Language Arts“ 


English® 


Algebra IP 


Social Science® 


American History® 


Algebra 2 or Geometry® 


Environmental Science® 


Chemistry® 


Travel and Tourism II*’ 


Floriculture*’ 


Foreign Language‘S 


Floriculture or Foreign Language*’ 
Health/Physical Education*’ 



“Required for graduation 
‘^Required for Academy 
‘^Elective 



As the Table illustrates, the Academy curriculum is a mix that straddles both courses 
required for graduation and elective courses that are taken as part of the Academy’s thematic 
program. Further courses may be either elective or (in the case of physical/health education in 
Philadelphia) additional requirements.^ 

In practice, however, there were variations among the schools examined in how 
effectively the course mix was managed. Schools with more experience operating in the 

^Here, too, variation is possible. For example, a student who completed the required number of health/PE 
courses may choose to take additional ones, which then would be regarded as electives. 
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Academy framework appear to have functioned more smoothly, though surprisingly, all of the 
schools experienced some common challenges. A closer examination of how the course selection 
and course-taking process played out in these settings provides some useful insights — 
especially for schools in the early stages of launching Academy programs. Three factors bear 
examination: 

a The kinds and sources of guidance students receive; 

□ The mix of elective and required courses in Academy programs; 

□ The effectiveness of information systems — including student rostering systems and 
forms. 

A. Kinds and Sources of Guidance 

Schools that adopt the Career Academy model have generally found it necessary to 
modify their counseling system, one of the longest established school processes, and with it the 
role of its guidance counselors. Historically, school guidance counselors have been both 
information sources and decision-makers where student course-taking was concerned. In many 
schools — including several examined in this study — counselors were responsible for 
overseeing students’ course selection, and approving rosters for the upcoming year. 

Unlike in traditional school settings where counselors were customarily responsible for 
scheduling, scheduling in Academy schools was negotiated between the counselor and the 
Academy coordinator. Counselors typically worked with a group of students who were more or 
less randomly assigned, often by alphabetical order of students’ last name. Under this 
arrangement, counselors monitored completion of courses required for graduation, but given the 
typically heavy caseload they carried — counselors might have 200 or more students assigned to 
them — they were often able to serve as little more than schedulers, or arbiters for disciplinary 
problems. 

In addition, counselors may not necessarily have been aware of the course requirements 
for a given Academy program — or even be informed that a student was in an Academy. The 
Career Academy model introduced a new wrinkle: The rosters of Academy students would have 
to be tracked to assure both compliance with graduation requirements and appropriate 
progression through the Academy curriculum. A portion of this latter responsibility would fall to 
the Academy coordinator, usually a teacher who would be accountable for overall management 
of the Academy program. 

Formally or informally, the coordinator determines that Academy students are tracking 
through the course sequence appropriately, ensuring that students complete courses needed to 
graduate. In fulfilling their mission as college preparatory programs, however. Academies were 
obligated to do more than simply ensure that students comply with graduation requirements. 
Academy coordinators had an interest in seeing to it that students who had the aptitude for 
college work also completed college-entry requirements. However, this task was complicated by 
the fact that students were required to complete additional coursework to enter state universities. 
(The added requirements for the study sites are summarized in Table 4.) 




14 



- 8 - 



Course Taking, Test Preparation, 
and Career Academy Programs 

Table 4 

State University Courses Not Included in High School Graduation Requirements 



Study Site 


Additional State Requirements 


Miami 


Foreign Language 


Oakland 


Mathematics 


Philadelphia 


Foreign Language 


St. Louis 


Foreign Language 


San Jose 


Mathematics 


Seattle 


Mathematics, Foreign Language 



Monitoring course progress among Academy students thus became a somewhat intricate 
undertaking, requiring knowledge of both normal high school requirements and the additional 
courses needed to apply to state universities. How Academy coordinators and school guidance 
counselors dealt with these somewhat overlapping (and potentially conflicting) roles determined, 
in part, the clarity of the guidance students received. 

The two schools with the oldest Academy programs modified the traditional approach in 
distinct and effective ways. In one of the schools, it was the Academy coordinator, not the 
guidance counselors, who was responsible for ensuring that students complete both graduation 
and Academy requirements. The coordinator was in a better position to track the progress of 
students were making in both Academy-program courses and in required courses, and they had 
the systems and forms to facilitate that monitoring task. For their part, guidance counselors 
focused on students’ post-secondary career and continuing education plans. 

In the other school, responsibility for course selection was shared between counselors and 
Academy coordinators. The counseling staffs student load was reorganized, with individual 
Counselors responsible for all the students in one Academy, rather than a portion of the 
alphabetized student list. This arrangement made it possible for counselors to work closely with 
the coordinator of their assigned Academy to help ensure that student rosters reflected progress 
both toward graduation and through the Academy curriculum.’ 

In both schools — indeed in all of the schools — informal coordination among teachers 
handling Academy-related subjects was an important tool for keeping track of students. 
However, the ideal arrangement, a cadre of teachers assigned to work solely in the Academy 
program was seldom to be found. Both because of a lack of common planning time and, as will 
be discussed in the following section, because of issues involved in rostering required courses, 
“Academy” teachers usually did not have an exclusive tie to the Academy program. While that 
informal connection made it possible for the teachers to be helpful in individual cases, it was too 
inconsistent to ensure that course taking by all Academy students would be carefully monitored. 



’The need for this kind of cooperation has been noted in earlier examinations of Academy scheduling issues. Cf. 
’’Scheduling Guide for Career Academies,” Berkeley, CA: Career Academy Support Network, 1 999. 
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In schools with more recently established Academies, the monitoring and rostering 
processes were more in flux. All of these schools used some combination of counselors and 
Academy coordinators in transitional settings that changed their operating styles as Academies 
were introduced or expanded. In only one of the schools, where the commitment to Academy 
programs was widely shared throughout the school, cooperative relationships among Academy 
coordinators and counselors were being developed. 

In other cases, however, the changes were not proceeding smoothly. In one of the newest 
Academy schools, the principal viewed creation of Academy programs as one lever for bringing 
about overall school improvement and change, but this vision encountered resistance. Counselors 
were responsible for rostering, but they were reluctant to reorganize their workload to 
accommodate the expanding number of Academies in the school. They claimed to find it 
difficult to identify the Academy students and to ensure that their course selection met both 
graduation and Academy course requirements. In another school, chronic turnover among 
teaching staff and principals continued to handicap efforts to operate Academies after more than 
five years of experience with the program. 

Additional complications arose when students failed courses — particularly required ones 
— or when they wished to take advanced placement (AP) or other electives and needed special 
roster help. Students failing courses might require provisional rosters since they might make up 
the failure in summer school. They might have to take courses out of sequence, or they might not 
be able to remain in an Academy at all. Similarly, students who wished to take AP courses, or 
otherwise tailor their rosters to suit their specific interests, further burdened the rostering process, 
by making it difficult to maintain the cohort-rostering so pivotal to the integrity of Career 
Academies. 

B. Mix of Elective and Required Courses in Academy Curricula 

The Academy curriculum consists of two types of courses: those required for graduation 
and electives. In the study schools, it was typically around the elective courses that the curricula 
were tailored to create Academy-specific content. Ideally, the traditional required courses — 
English, foreign language, mathematics, science, social science — would be modified somewhat 
to reflect the Academy theme.* 

In the study schools, however, the modification of traditional courses was unevenly 
implemented. In the school with the longest-established Academy programs, the required 
subjects had hardly been tailored to Academy themes at all. The belief there was that the core 
subjects ought to be taught strictly on their own terms — “English is English,” is how one 
respondent put it. In the other schools, the tailoring of core subjects was minimal and 
fragmentary. 

Since the Academy content is mostly concentrated in electives. Academy students 
wanting added variety in their rosters have limited rostering choices. Their elective slots are 
mostly determined by the required Academy courses. Their main rostering alternatives therefore 
often center on the available “sections” of required courses. 



*See Tables 2 and 3. 
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For example, after rostering Academy electives, a student following an Academy 
sequence who wishes to take an AP English course may have no choice but to take the only 
section of junior math that fits her schedule. As a result, she will likely miss taking the math 
section her Academy peers are rostered into. Similarly, a student who has failed a required 
English or math course is almost certain to take that course with an unrelated group of students, 
either in the summer or the following year. 

Compounding the problem of limited course selection in the study schools was a shortage 
of teachers for science and math courses. Their scarcity limited the number of advanced courses, 
and sometimes even required courses, that were rostered. That, in turn, constrained Academy 
students’ choices. If the only section of “elementary functions” conflicted with a required 
Academy course, the student would have little choice but to roster the Academy course, even if 
he or she were interested in the math course. 

In general, students in the first year of an Academy sequence were more routinely 
“cohort-rostered,” that is, grouped together in courses.^ The feasibility of course-rostering for 
Academy students dwindled in the junior and senior years. Individual student preferences, 
limitations in course availability, and the need to repeat failed courses caused Academy students 
to be dispersed into whatever sections of required courses they could fit into their rosters. Only 
in the Academy-related electives were they likely to be brought together as a cohort.'® 

This gradual dispersion did not of itself increase the risk that students might fail to 
complete graduation requirements. It did mean that upper-level Academy students would be 
spread across several sections of a required course rather than being cohort-rostered. That made 
it harder (and less urgent) for Academies to try tailoring required courses to the Academy theme. 
Instead, the established departments — English, math, social science — regulated these courses, 
as they traditionally had. Their natural concern was to ensure that individual sections of these 
courses adhered to districtwide content standards. 

In the study sites, therefore, individual sections of upper-level required courses tended to 
be generic and untailored. Consequently the Academy experience was limited in the amount of 
specialization and coherence it could achieve. Rather than affecting students’ ability to complete 
graduation requirements, it seemed, the Academy curriculum was itself likely to be affected by 
the primacy of those requirements. 

C. Effectiveness of Information Systems 

The final factor affecting course-taking was the information systems in the schools. In all 
cases, a computerized rostering system of some sort was in use. However, the accessibility of 
those systems, their responsiveness to the demands posed by Academy rostering, and how 

’Even in an Academy that started in the tenth grade, students might, because of previous course failures, be 
dispersed across both 9'*' and 1 O'*' grade English, math and social science. Only in Academy programs beginning in 
the 9''' grade could consistent grouping be accomplished in the first year. 

'®A related issue was the need, at the building level, to maintain appropriate teacher-student ratios, consistent 
with the teaching complement at the school. Thus, in many cases, small courses for Academy-only students would 
be precluded because of the need to use available teacher resources efficiently and equitably. 
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effectively they were used by the involved parties significantly affected the rostering process — 
indeed, had a real effect on the degree of choice afforded to students. 

The most effective system was found in one of the established Academy schools, where 
meeting the rostering needs of individual students was a major priority. Counselors and 
Academy coordinators were trained to use the system, had ready access to up-to-date information 
on student course completions and options, and could work flexibly to adapt schedules as 
needed. 

Teachers and students reported that student choices and preferences could usually be met 
and that students could be encouraged to diversify their course selections with reasonable 
expectations of getting their choices. It was easier to keep Academy students grouped in required 
courses, though a fair amount of dispersion remained even in this favorable environment. 

Information systems were in place at the other Academy program schools, but their 
effectiveness in meeting student rostering needs varied. Because the information systems had 
originally been designed to serve guidance counselors as the rostering “gatekeepers,” it was 
typically, guidance counselors — not the Academy coordinators — who had better access to 
computerized student data. Student requests were difficult to accommodate, and it was 
commonly harder to tailor the rostering process to the needs of Academy students." The least 
effective setting was in a school where rostering, though computerized, appeared to be beyond 
anyone’s direct control. Neither Academy coordinators nor counselors felt they had much say in 
how individual students’ rosters would turn out. 

D. Course-Taking; Conclusions 

The findings from the school visits provide no evidence that Academy curricula hinder 
students from completing courses required for graduation. To the contrary, all of the schools 
examined are intensively focused on ensuring that students meet course graduation requirements 
as their paramount concern. Adherence to Academy curriculum requirements, while not ignored, 
was clearly a secondary consideration. 

It should not be concluded, however, that the administrative process works flawlessly. To 
the contrary, the overburdened counseling apparatus found in the study schools was likely to 
exhibit mechanical and superficial rostering choices that afforded little time for real guidance 
and counseling. Such individual attention counselors were able to give was more likely to be 
conferred on students who were more engaged rather than on those who were in greater need of 
assistance. Because they had more interaction with their Academy coordinator. Academy 
students in the schools examined for this study were more likely than the non-Academy students 
to receive guidance and careful advice about rostering choices. 

There was no conflict concerning this issue between Academy program staff and other 
teaching or administrative staff in the schools. The opposite was found to be the case. Academy 
coordinators and teachers were every bit as anxious as the rest of the school to see their students 
graduate. They readily accepted the need to accommodate graduation requirements. Since, in 

"in one school visited, the counseling staff was bluntly criticized by the principal for resisting efforts to align 
the counseling and rostering process with the newly established Academies. 
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fact, the Academy curricula had usually been designed around those requirements in the first 
place, ensuring compliance required no extraordinary steps. 

Academy coordinators and teachers were every bit as anxious as the rest of the school to 
see their students graduate, and they readily accepted the need to accommodate graduation 
requirements. The preoccupation that students meet graduation requirements appeared to have an 
effect — and not necessarily a positive one — on the Academy programs. It appeared to increase 
the tendency for courses required of both Academy and non-Academy students to become more 
homogeneous. Academy students would be more inclined to register in core courses or, 
especially in the ll'*’ and 12'’’ grades, in courses that meet college-entrance requirements, even if 
that meant they would be taking those courses apart from their Academy peers. 

This concern manifests itself especially in schools that are electing to go “All-Academy,” 
that is, to be composed only of Academy programs. The logistical challenge of offering separate 
Academy-oriented core courses to six or seven separate Academy groups is considerable. As 
students reach the upper grades when their course-taking paths begin to diverge, the goal of 
maintaining any thematic coherence beyond the specified Academy elective courses appears 
unattainable, particularly in view of faculty limitations and the need to meet class-size 
requirements. 

Schools more experienced with Academy programs were better able to respond to this 
problem. The two study schools with the longest-operating programs managed to develop 
processes that balanced Academy needs, graduation requirements, and individual student 
interests fairly effectively. Schools that were newly implementing Academies were still working 
through rostering issues. Though there appeared to be little risk that students would fail to take 
required courses, there was evidence that the goal of keeping Academy students together for 
required courses might be more limited. 

III. High-Stakes Testing 

The debate over whether performance standards can be used effectively to assess school 
quality is widespread and still unresolved. There continues to be concern over how such 
standards can be developed, whether national testing requirements are a useful or appropriate 
goal, how they affect local autonomy in framing curriculum, and how test results should be 
interpreted and used. 

As that debate goes on, the use of student testing as a way of assessing school 
performance continues to grow. Some 40 states now have testing programs instituted to serve 
two key purposes. First, they are intended to gauge student attainment and competency in such 
essential areas as reading and literacy, computational and mathematics knowledge, the physical 
and social sciences. Secondly, and more controversially, the tests are used to assess school 
performance.'^ The test scores for an entire school district, or individual schools within a district, 
are now frequently published in newspapers on an annual basis, and have become the implicit 
standard forjudging whether a school, or a public school system, is doing its job. 



‘^Until recently norm-referenced tests, such as the Stanford Achievement Test, were used for this purpose. 
Recently, however, state-designed and required tests have begun to supplement or replace them. 
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At the secondary-school level, the new tests have been superimposed on schools whose 
students are already subject to high-stakes exams, like the PSAT, SAT, and ACT, which have 
long been a rite of passage for high school youth. But in an environment, now heavily dominated 
by this stress on high-stakes testing, two major questions arise as to how Academy students can 
be expected to perform: (1) Does the specialized thematic focus of Academies detract either from 
their performance on these tests or even from their interest in taking them? (2) Are Academy 
students as well prepared as students in traditional school settings to take the tests? 

To answer these questions it is helpful, first, to look separately at the separate kinds of 
tests given to students. There are four basic kinds: 

1. Standardized (commonly norm-referenced) tests, usually administered under the 
auspices of the local school district; 

2. Competency-based assessment tests, usually developed by and administered 
through the requirements of the state, and including high school exit examinations in 
states that mandate them;’^ 

3. Traditional college entrance achievement tests (PSAT, SAT and ACT), taken 
voluntarily by students; and 

4. Advanced Placement examinations, which assess student competence in individual 
subjects. 

The first two are required of all students either at specified grades (e.g., seventh, ninth, 
eleventh), or, in some cases, administered annually. All students are encouraged to sit for college 
achievement tests; however, participation is voluntary. Advanced Placement tests are taken only 
by better-achieving students to demonstrate superior performance or knowledge in specific 
subject areas. 



'^Two of these tests involved hand grading. One had a writing component; the other was designed around a set 
of fairly complex problem items, whose solution required several steps and some explanation of the steps taken. 
Grading those parts was time-consuming, and it was often several months before the test results became available. 
The test format was also controversial in those sites. 
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The following table indicates the kinds of tests administered in the study sites. 

Course Taking, Test Preparation, 
and Career Academy Programs 

Table 5 



Achievement and Proficiency Tests in the Study Sites 



Site 


State-Designed 
Achievement Test 


Other Local Test 


Minimum Score 
Needed to Graduate? 


Philadelphia 


Yes 


SAT9 


No 


St. Louis 


Yes 


No 


No 


San Jose 


Yes“ 


No 


No 


Oakland 


Yes“ 


Yes*” 


Yes*^ 


Seattle 


Yes 


Yes*^ 


No** 


Miami 


Yes 


No 


Yes 



“California uses the SAT9 
'’For ninth-graders only 

“Local proficiency standards must be met for graduation 
‘‘Passing scores will be required by 2008. 

In three of the study schools, the institution of a state test was a recent development 
within the past two years. In two other cases, the state test was just being introduced on a pilot 
basis. In one site students were required to attain a minimum score in order to graduate, and in 
Oakland minimum proficiency standards had to be achieved. 

A. Testing and Schooling 

Before examining the Academy context, a broader observation is warranted. In all of the 
study schools, the competency testing requirements were regarded as anything from a nuisance 
to a simplistic and misleading approach to measuring school performance. The frustration and 
negative opinions spread from principals to staff to students. Some of the specific reasons for this 
widespread dislike will be discussed below. A broader concern, though, is how much does the 
negative opinion about testing programs affect the dedication with which teachers administer 
them and students take them? 

B. High Stakes for Whom? 

Students interviewed for this study usually made a qualitative distinction between 
college-related testing (i.e., ACT, SAT and AP) and the state-promulgated competency tests. The 
former they correctly saw as high-stakes tests for them. As for the latter, students acknowledged 
that it was “important for my school” to have good test results, but with the exception of 
competency tests they had to pass in order to graduate, they did not perceive that their individual 
self-interest was directly involved. Not surprisingly, the college-related tests received a more 
intensive effort than the competency tests on the students’ part. “I make sure I get through my 
SATs okay,” said one student, who found it harder to “get up for those other tests.” This view 
was consistent with several teachers’ observation that, by spring (typically the season when state 
tests were administered), many of the upper-class students were already “tested out.” 
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C. Mixed Messages About Testing 



The orthodox message delivered by teachers to students was that doing well on the 
competency tests mattered because it affected the school’s reputation. One teacher reported she 
told students, “It [the test results for the school overall] makes a difference in how much money 
the school gets.” In interviews, however, teachers questioned the soundness and validity of the 
tests, and were highly critical of their use as a summary “bottom-line” measure of school 
performance. Given their consistently negative opinions, it seems unlikely that students would 
fail to note the contradictory messages being sent, which may well have contributed to their 
discounting the importance of these tests. 

D. Duration and Disruption 

The chief complaints heard both from staff and students were that testing periods last too 
long and that they are disruptive of school schedules. Typically, state competency tests might 
officially be given over a period of two weeks. However, counting time beforehand to organize 
the testing schedule and juggle teaching and proctoring assignments and adding time afterward 
to complete make-up tests, especially for students who had been absent,'^ the actual time spent 
testing could amount to a month. During that period, staff and students agreed, regular 
schooling generally failed to happen. 

Though the deadlines and schedule generally were promulgated with ample lead time, 
they were not always well thought out. In two schools, the competency testing was scheduled to 
occur in the same week as AP tests. Teachers reported that they encouraged their students to 
prepare for the AP tests and, if necessary, to skip the competency tests in order to complete AP 
testing. The result was that test results for these schools were biased by the absence of some of 
their highest-performing students. Policy at another study school (discussed further below) 
sharply curtailed extracurricular activity in the six weeks prior to the testing period so that 
students could participate in test preparation classes. This restriction had a materially negative 
effect on the Academy, whose curriculum required frequent visits to local Academy-related 
businesses. The visits had to be suspended during the pre-test period and other coursework 
reshuffled as a result. 

These findings, anecdotal and based on a limited number of schools, nonetheless were 
strong and consistent. They suggest that, however valid a measure the tests are and whatever 
weight is attached to the results, the negative attitudes toward testing evident in schools reduce 
the importance of the tests in the eyes of students and teachers and may undercut efforts to 
encourage students to take them seriously. 



‘■’ll! one school, the principal reported that truancy rates shot up appreciably during the testing period. So acute 
was the problem that the school had to institute a crash program to locate and test enough students so that its overall 
completion rate would be considered valid in computing test scores. In a school in another district visited for this 
study (though not one of the study schools), the entire testing program had to be repeated because insufficient 
numbers of students had taken the test. 

'^One month was the most frequent estimate voiced by students and staff at the study schools regarding the 
effective length of the testing period. In one school, though staff reported having lost six weeks of the previous 
academic year to test administration. 





E. Preparing Students; Academy and Non-Academy 



The institution of state tests was so recent in three of the study schools that no plans were 
in place to provide preparatory classes or other help for students. In two others schools, 
discussions had taken place about what kind of assistance to provide, but no concrete steps had 
been taken. The administration at a sixth school had determined that no additional help could be 
offered, apart from tutoring and homework help already available to the student body generally. 

The schools examined for this study usually had an extensive history of offering 
themselves to linking students to preparatory courses to help ready the test takers for college 
placement exams. Some of the schools also offered tutoring and homework help programs, and 
after-school assistance programs were also offered. Administrators and teachers seemed daunted 
by the prospect of trying to add further preparatory help in response to the new tests, given the 
programs they already were involved with. 

Academy programs within the schools did not take special responsibility for preparing 
their students for the new overlay of state tests. In only one of the Academy programs examined 
in this study had special preparatory courses or sessions been developed to boost the 
performance of Academy students in state tests. The only help the others offered was to guide 
students toward resources that already existed. 

There is ample evidence, though, that Academy staff were concerned not solely about test 
performance but also about the post-graduation outcomes of their students. Academy 
coordinators reported that they used their best skills of exhortation to encourage students to take 
and do well on tests, and they enlisted other Academy teachers provide encouragement, as well. 
In the established programs, where roles were better developed, coordinators professed that their 
personal relationship with students made a difference in the willingness of students to take the 
testing seriously. 

In the newer programs, coordinators found it much more difficult to determine how much 
weight their powers of personal persuasion carried. However, coordinators and Academy 
teachers expressed consistently their desire to see students achieve. They provided them with 
ongoing encouragement to prepare for and take college entrance tests, helped adjust schedules so 
students could take AP or other useful electives (subject to the kinds of constraints discussed 
earlier regarding rostering), and generally stressed the value of post-secondary education for all 
their students. 

Coordinators looked to the school administration or to the traditional academic 
departments as the appropriate source of preparatory help for tests. Since the tests stressed basics 
like English, math, and social science, it seemed natural that the established departments would 
provide added assistance. Since the schools as a whole, or individual departments, often did not 
conclude that added preparation was needed, the Academy programs implicitly followed suit. 

That view of where supplemental test preparation help should originate was widely 
shared throughout the schools, with the exception of one school district alluded to above. It that 
instance, the school district developed a district-wide policy for preparing students to take state 
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assessment tests. Schools in that district were required to take a number of steps to boost student 
performance, which in addition to restricting extracurricular activities, included initiatives to: 

□ Revamp curricula to conform to the emphases in the state test; 

u Design curricular tests so that, to the extent possible, they would mirror the format of 
the standardized test; 

□ Provide preparatory sessions for students in the weeks preceding test administration.'^ 

In this school both Academy and non-Academy students received an intensive regimen of 
pre-test preparation, as well as a curriculum oriented to reflect the content areas of the state test. 

Here, as in other study schools, there was no concern about the emphasis of the state tests 
impinging on or distorting the content of the Academy curricula.'^ Since the main Academy- 
specific material was covered in elective courses, rather than required subjects. Academy 
curricula would be little affected unless substantial changes in required subject content occurred. 

F. High-Stakes Testing; Conclusions 

The trend among states to gauge school performance through the use of standardized tests 
and hold faculty and administration accountable for results intensifies, even as the debate 
continues about the value of this approach. Certainly it has raised the stakes for students who 
must take this new overlay of performance test. 

In the study schools, any effects of test taking on Academy programs (or vice versa) were 
overshadowed by the generally negative opinion that increased state testing regimens engendered 
within schools and by the disruptions to school operations those testing requirements produced. 
Teachers and students alike found these tests burdensome, students sometimes resisted taking 
thepi, and there was broadly expressed skepticism about their value and their validity. 

Academies instituted no special programs to help prepare students to take and do well on 
these tests. The one exception to this resulted from sweeping district-wide policies that required 
specialized test preparation courses, changes to curriculum, and even specific teaching practices 
(for example, the formatting of tests) in order to better align the classroom experience with the 
format and content of the state test. Though Academy students benefited (as did other students in 
the school) from this effort, the Academy program suffered in other ways. 

IV. Reflections and Recommendations 

The purpose of this study was to determine how Academy programs performed in the 
context two important current themes in public education: concern that students complete 

'^This state test had spawned a mini-boom in academic publishing. Two different private firms had developed 
practice and preparatory materials in support of the state test, which were being purchased and used by school 
districts across the state. 

'’The limitation on extracurricular activities in this school did, as noted previously, limit some of the private 
sector visiting and interviewing that is emphasized in the Academy curricula. 



courses required for graduation and college entry, and the increasingly heavy emphasis on high 
stakes tests. 

The findings presented here, though they are qualitatively derived and based on a limited 
number of study schools, suggest that, at least regarding these two issues, the organization and 
behavior of individual Academy programs caimot be considered in isolation. Rather they must be 
viewed in the context of how the schools as a whole (and, in one case, district) behave. 

Academies are effective in ensuring that students complete required courses, but school 
systems and individual high schools already monitor course completion as a fundamental part of 
their operations. The ability of Academies to ensure course completion is largely determined by 
the schools in which they operate. They might improve on it marginally, but it would be unlikely 
for their performance to be weaker than that of their host school. 

It is more commonly the case that the complexities of course scheduling affect Academy 
programs. The challenges of rostering 1,500 or more students in a high school into their required 
courses are considerable, and they are magnified by students who repeat courses, by those who 
elect to take individualized sequences of electives or required courses, and by limitations in the 
number and qualifications of faculty. 

While most Academy programs are coherent and robust in the 9**’ and 10'*’ grades, many 
lose their distinctiveness in the later high school years when it becomes increasingly difficult to 
keep intact the group identity of an Academy class as it advances into upper-year classes. 
Similarly, it becomes difficult to retain the Academy-specific elements of, say, an English or 
math course, when significant numbers of students taking that class may not belong to the 
Academy. Faced with the geometrically expanding roster demands imposed by more Academies, 
the challenges of varying the illustrative details (and to some degree the core content) of required 
courses will prove ever more difficult to meet.'* 

The challenges grow as schools adopt an “all-Academy” structure. To meet them will 
require careful attention to the organizational issues they raise. At the least, counseling staff must 
be deployed in a way that ensures that students in the distinct Academy strands are coherently 
rostered. Counselors and Academy coordinators will need to have carefully delineated roles 
when it comes to advising students so that the interrelated tasks of career guidance, course 
selection, and decisions regarding continuing education are handled smoothly. 

A stable and dedicated cadre of school faculty, who can work flexibly across the 
requirements of different Academies while remaining faithful to their core subjects, would help 
mitigate the problems, even if much of the work they do in that regard is informal. Regrettably, 
the experience of the study schools where faculty turnover and reassignment are common 

'*A related problem is that a school’s population of students may not sort tidily into a finite number of 
Academies. In two of the study sites, there was some reference to a generic “liberal arts” academy — a catchall place 
for students who expressed no interest in the other academy programs. 



suggests that such stability is rare. In only two of the schools did Academies have a relatively 
stable core of faculty, and both did demonstrate the benefit of such an arrangement.^^ 

The coherence of Academies, whether in “wall-to-wall” settings or not, may also be 
affected if alignment of curricula with test contents assumes priority importance. The risk is 
twofold. First and most obviously, required courses may be reconfigured in ways that make it 
difficult to adapt them to the Academy theme and activities. In the one site examined for this 
study where alignment was a priority, the possibility of teachers adopting a pure “teach to the 
test” stance was raised. It is too early to predict how that will play out, but the risk seems real. 

A more subtle effect may be that the salience of the Academy approach is largely washed 
out by concern over test content. The best Academies undoubtedly are effective in providing 
educational experiences that show well in standardized test scores. However it may look more 
expedient to prepare students by “aligning curriculum with test standards,” rather than by 
supporting variant educational experiences lacking formal congruence with the state test format 
or emphases. 

As public education moves through its current preoccupation with core courses and test 
results. Academy programs, especially those just now getting started, will face numerous 
challenges. While they cannot all be anticipated, this study would suggest several steps for 
Academy programs, and their supporters, to consider. 

u Increased attention should be given to building and maintaining a coherent and 
stable educational experience for students in each Academy. 

A responsive arrangement of counseling staff, along with flexible and adaptive rostering 
procedures, is essential. Special attention must be given to ensure that students complete required 
courses seems unnecessary. Since schools are already organized to maintain that emphasis. 
Academies need only work to see to it that the basic systems do not permit students to fall 
through the cracks. 

The ideal arrangement, cohort-rostered courses taught by a dedicated faculty, should be 
sought where possible, though the prevailing emphasis on standardized test results and required 
courses may work against its achievement. Short of that, more consistent attention to cross- 
subject projects may be one of the most effective techniques available for achieving a coherent 
curriculum. The interdisciplinary projects can embody the aims of both Academy and traditional 
subjects, and they can be implemented successfully even where faculty connection to the 
Academy is informal. 

These conclusions suggest that the expansion of Academy programs should proceed 
deliberately. Each new program a school adds will require time to work through the institution’s 
existing administrative arrangements and will present problems requiring attention and effort to 
resolve. While it may be laudable to seek school change through Academy programs, the 
programs must not be too casually installed at the risk of ending up with Academies in name 

’^Only one of the study schools had common planning time for Academy faculty. One other (a long-established 
program) had established routines for keeping faculty in touch regarding individual students, as necessary. 
Otherwise, coordination among Academy faculty was casual and limited. 
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only. These would devolve into a sequence of minor electives, instead of developing the 
distinctiveness and integrated substance that mark the best Academy programs. 

u Academy programs should explore more ways to demonstrate the unique 
attainments of their students. 

The current emphasis on testing and accountability increases the risk that educational 
perspective will become too narrow. Academy students should be expected to meet educational 
attainment and academic quality standards without placing paramount importance on test scores 
as an end in themselves. 

Academy programs should push for more varied measures to judge the attairunents of its 
students. Whether through portfolio assessments, special projects, community (and especially 
industry) recognition or other academic awards. Academies need to find ways to illustrate the 
kinds of special achievements they are geared to produce and to demonstrate that their approach 
to education can meet both testing standards and other measures of personal growth and 
achievement. 

□ Academies should find ways to ensure that their students succeed in the 
standardized testing environment. 

To date, both schools and Academies (as regards sites examined for this study) have been 
generally slow to respond to state-mandated testing and the accountability perspectives it 
reflects. Academy programs face challenges enough simply launching themselves and 
developing their own distinctive identities. Significant additional efforts required to help students 
respond to a new layer of tests may overtax their limited staff resources. 

It appears likely, however, that the primacy of performance testing will remain a reality 
for the foreseeable future. To prove their value in that envirorunent. Academies will have to 
extend themselves to devise innovative methods consistent with their distinctive educational 
approach and provide their students with the preparation they need to make a credible showing 
on proficiency tests of all kinds. 





Course Taking, Test Preparation, 
and Career Academy Programs 



Exhibit 1 



Overview of Research Framework for Course-Taking Study 
Topic Key Issues 



Requirements 



Li State/local graduation requirements 
□ Promulgation by local school district 
u Academy requirements — add-ons 



Sequencing 



Communication 



□ State/local requirements 

u Promulgation by local school district 

□ Academy requirements — how they may interact with existing 
school district standards 

□ Special circumstances — course failure, desire for special courses, 
limited course availability, etc. 

u How teachers, counselors and Academy coordinators are informed 
about overall course-taking requirements 

□ How counselors are informed about Academy requirements 

□ How students are informed about course-taking requirements 



Guidance 



Monitoring and 
Enforcement 



Grades 



Accreditation 



u What information and advice students are given 

□ When information and advice are given 

□ School staff responsibilities for approval of course selection among 
Academy students 

□ How roster conflicts or irregularities are addressed 

□ How appropriateness and correctness of course-taking are tracked 

□ Actual student performance in school work 

□ How school performance may affect sequencing and course-taking 

□ Course development standards and formal recognition of Academy 
courses within school district 

□ Approval process for Academy courses 
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Course Taking, Test Preparation, 
and Career Academy Programs 



Exhibit 2 



Overview of Research Framework for High-Stakes Test Study 



Topic 

Types of Tests 



Key Issues 

□ Required for graduation (state/local achievement tests) 
u General Proficiency (e.g., Stanford) 
u College-related (e.g., state school requirements, SAT, ACT) 



Timing 



u Frequency of testing 
□ Sequencing of testing 
u Coordination of test activities 



Test Preparation Activities 



□ Formal preparation activities 

u Staff responsibilities for test preparation 

□ School -wide versus Academy preparatory activities 

□ Availability of outside assistance 

□ Specialized instruction/tutoring 



Communication and Guidance 



u Extent of guidance and information regarding testing 
requirements 

u Career and post-secondary guidance 
□ Extent of information about test-taking assistance 
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Recent Publications on MDRC Projects 



Note; For works not published by MDRC, the publisher’s name is shown in parentheses. With a few exceptions, this 
list includes reports published by MDRC since 1999. A complete publications list is available from MDRC 
and on its Web site (www.mdrc.org), from which copies of MDRC’ s publications can also be downloaded. 



Education Reform 

Accelerated Schools 

This study examines the implementation and impacts 
on achievement of the Accelerated Schools model, a 
whole-school reform targeted at at-risk students. 

Evaluating the Accelerated Schools Approach: A 
Look at Early Implementation and Impacts on 
Student Achievement in Eight Elementary Schools. 
2001. Howard Bloom, Sandra Ham, Laura Melton, 
Julienne O’Brien. 

Project GRAD 

This evaluation examines Project GRAD, an 
education initiative targeted at urban schools and 
combining a number of proven or promising reforms. 

Building the Foundation for Improved Student 
Performance: The Pre^Curricular Phase of Project 
GRAD Newark. 2000. Sandra Ham, Fred Doolittle, 
Glee Ivory Holton. 

Career Academies 

The largest and most comprehensive evaluation of a 
school-to-work initiative, this study examines a 
promising approach to high school restructuring and 
the school-to-work transition. 

Career Academies: Early Implementation Lessons 
from a 1 0-Site Evaluation. 1996. James Kemple, 
JoAnn Leah Rock. 

Career Academies: Communities of Support for 
Students and Teachers — Emerging Findings from 
a 10-Site Evaluation. 1997. James Kemple. 

Career Academies: Building Career Awareness and 
Work-Based Learning Activities Through Employer 
Partnerships. 1999. James Kemple, Susan 
Poglinco, Jason Snipes. 

Career Academies: Impacts on Students' 

Engagement and Performance in High School. 
2000. James Kemple, Jason Snipes. 



SchooI-to-Work Project 

A study of innovative programs that help students 
make the transition from school to work or careers. 

Home-Grown Lessons: Innovative Programs Linking 
School and Work (Jossey-Bass Publishers). 1995. 
Edward Pauly, Hilary Kopp, Joshua Haimson. 
Home-Grown Progress: The Evolution of Innovative 
School-to- Work Programs. 1 997. Rachel Pedraza, 
Edward Pauly, Hilary Kopp. 

Project Transition 

A demonstration program that tested a combination of 
school-based strategies to facilitate students’ 
transition from middle school to high school. 

Project Transition: Testing an Intervention to Help 
High School Freshmen Succeed. 1 999. Janet Quint, 
Cynthia Miller, Jennifer Pastor, Rachel Cytron. 

Equity 2000 

Equity 2000 is a nationwide initiative sponsored by 
the College Board to improve low-income students’ 
access to college. The MDRC paper examines the 
implementation of Equity 2000 in Milwaukee Public 
Schools. 

Getting to the Right Algebra: The Equity 2000 
Initiative in Milwaukee Public Schools. 1999. 
Sandra Ham, Erica Walker. 

Education for Adults and Families 

LILAA Initiative 

This study of the Literacy in Libraries Across 
America (LILAA) initiative explores the efforts of 
five adult literacy programs in public libraries to 
improve learner persistence. 

So I Made Up My Mind: Introducing a Study of Adult 
Learner Persistence in Library Literacy Programs. 
2000. John T. Comings, Sondra Cuban. 

“/ Did It for Myself: Studying Efforts to Increase 
Adult Learner Persistence in Library Literacy 
Programs. 2001. John Comings, Sondra Cuban, 
Johannes Bos, Catherine Taylor. 
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Toyota Families in Schools 

A discussion of the factors that determine whether an 
impact analysis of a social program is feasible and 
warranted, using an evaluation of a new family 
literacy initiative as a case study. 

An Evaluability Assessment of the Toyota Families in 
Schools Program. 2001 . Janet Quint. 

Opening Doors to Earning Credentials 

An exploration of strategies for increasing low-wage 
workers’ access to and completion of community 
college programs. 

Opening Doors: Expanding Educational 
Opportunities for Low-Income Workers. 2001. 
Susan Golonka, Lisa Matus-Grossman. 

Effects of Welfare and Antipoverty 
Programs on Children 

Next Generation Project 

A collaboration among researchers at MDRC and 
several other leading research institutions focused on 
studying the effects of welfare, antipoverty, and 
employment policies on children and families. 

How Welfare and Work Policies Affect Children: A 
Synthesis of Research. 2001 . Pamela Morris, 

Aletha Huston, Greg Duncan, Danielle Crosby, 
Johannes Bos. 

How Welfare and Work Policies Affect Employment 
and Income: A Synthesis of Research. 200 1 . Dan 
Bloom, Charles Michalopoulos. 

Minnesota Family Investment Program 

An evaluation of Minnesota’s pilot welfare reform 
initiative, which aims to encourage work, alleviate 
poverty, and reduce welfare dependence. 

Reforming Welfare and Rewarding Work: Final 
Report on the Minnesota Family Investment 
Program. Volume 2: Effects on Children. 2000. 

Lisa Gennetian, Cynthia Miller. 

Reforming Welfare and Rewarding Work: A 
Summary of the Final Report on the Minnesota 
Family Investment Program. 2000. Virginia Knox, 
Cynthia Miller, Lisa Gennetian. 

Canada’s Self-Sufficiency Project 

A test of the effectiveness of a temporary earnings 
supplement on the employment and welfare receipt of 
public assistance recipients. 

The Self-Sufficiency Project at 36 Months: Effects on 
Children of a Program That Increased Parental 



Employment and Income (Social Research and 
Demonstration Corporation, Ottawa, Canada). 

2000. Pamela Morris, Charles Michalopoulos. 

National Evaluation of Welfare-to-Work 
Strategies 

Conceived and sponsored by the U.S. Department of 
Health and Human Services (HHS), with support 
from the U.S. Department of Education (ED), this is 
the largest-scale evaluation ever conducted of 
different strategies for moving people from welfare to 
employment. 

Do Mandatory Welfare-to-Work Programs Affect the 
Well-Being of Children? A Synthesis of Child Re- 
search Conducted as Part of the National Evalua- 
tion of Welfare-to-Work Strategies (HHS/ED). 
2000. Gayle Hamilton. 

Teen Parents on Welfare 

Teenage Parent Programs: A Synthesis of the Long- 
Term Effects of the New Chance Demonstration, 
Ohio ’s Learning, Earning, and Parenting (LEAP) 
Program, and the Teenage Parent Demonstration 
(TPD). 1998. Robert Granger, Rachel Cytron. 

Ohio’s LEAP Program 

An evaluation of Ohio’s Learning, Earning, and 
Parenting (LEAP) Program, which uses financial 
incentives to encourage teenage parents on welfare to 
stay in or return to school. 

LEAP: Final Report on Ohio 's Welfare Initiative to 
Improve School Attendance Among Teenage 
Parents. 1997. Johannes Bos, Veronica Fellerath. 

New Chance Demonstration 

A test of a comprehensive program of services that 
seeks to improve the economic status and general 
well-being of a group of highly disadvantaged young 
women and their children. 

New Chance: Final Report on a Comprehensive 
Program for Young Mothers in Poverty and Their 
Children. 1997. Janet Quint, Johannes Bos, Denise 
Polit. 

Parenting Behavior in a Sample of Young Mothers in 
Poverty: Results of the New Chance Observational 
Study. 1998. Martha Zaslow, Carolyn Eldred, 
editors. 
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MDRC Working Papers on 
Research Methodology 

A new series of papers that explore alternative 
methods of examining the implementation and 
impacts of programs and policies. 

Building a Convincing Test of a Public Housing 
Employment Program Using Non-Experimental 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howard Bloom, 

Estimating Program Impacts on Student Achievement 
Using "'Short” Interrupted Time Series. 1999. 
Howard Bloom. 

Using Cluster Random Assignment to Measure 
Program Impacts: Statistical Implications for the 
Evaluation of Education Programs. 1 999. Howard 
Bloom, Johannes Bos, Suk-Won Lee. 



Measuring the Impacts of Whole School Reforms: 
Methodological Lessons from an Evaluation of 
Accelerated Schools. 2001. Howard Bloom, 

The Politics of Random Assignment: Implementing 
Studies and Impacting Policy. 2000. Judith Gueron. 

Modeling the Performance of Welfare-to-Work 
Programs: The Effects of Program Management 
and Services, Economic Environment, and Client 
Characteristics. 2001. Howard Bloom, Carolyn 
Hill, James Riccio. 

A Regression-Based Strategy for Defining Subgroups 
in a Social Experiment. 2001 . James Kemple, Jason 
Snipes. 

Extending the Reach of Randomized Social 
Experiments: New Directions in Evaluations of 
American Welfare-to- Work and Employment 
Initiatives. 2001. James Riccio, Howard Bloom. 
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About MDRC 



The Manpower Demonstration Research Corporation (MDRC) is a nonprofit, 
nonpartisan social policy research organization. We are dedicated to learning what 
works to improve the well-being of low-income people. Through our research and 
the active communication of our findings, we seek to enhance the effectiveness of 
social policies and programs. MDRC was founded in 1974 and is located in New 
York City and San Francisco. 

MDRC’s current projects focus on welfare and economic security, education, and 
employment and community initiatives. Complementing our evaluations of a wide 
range of welfare reforms are new studies of supports for the working poor and 
emerging analyses of how programs affect children’s development and their 
families’ well-being. In the field of education, we are testing reforms aimed at 
improving the performance of public schools, especially in urban areas. Finally, our 
community projects are using innovative approaches to increase employment in 
low-income neighborhoods. 

Our projects are a mix of demonstrations field tests of promising program 

models and evaluations of government and community initiatives, and we 
employ a wide range of methods to determine a program’s effects, including large- 
scale studies, surveys, case studies, and ethnographies of individuals and families. 
We share the findings and lessons from our work — including best practices for 
program operators — with a broad audience within the policy and practitioner 
community, as well as the general public and the media. 

Over the past quarter century, MDRC has worked in almost every state, all of the 
nation’s largest cities, and Canada. We conduct our projects in partnership with state 
and local governments, the federal government, public school systems, community 
organizations, and numerous private philanthropies. 
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